Skip to content

Enable gradient explanations in client - #34

Merged
JGagacheva merged 38 commits into
mainfrom
jg-enable-explain
Nov 3, 2026
Merged

JGagacheva merged 38 commits into
mainfrom
jg-enable-explain

Conversation

@JGagacheva

@JGagacheva JGagacheva commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Enable gradient explanations for kumo-tabular and kumo-relational, one per scored row.

  • explanation in outputs is accepted on both models with no row limit. The one-row/one-entity checks and the old 1,000-row context rule for relational explanations are removed; the SDM NIM
    (structured-data-nims 0a8230d) explains every predict row.
  • result.explanations maps each row's key (id; (id, forecast_step) for forecasts;
    row_index when the request has no id column) to its explanation, kept out of to_pandas().
    Replaces result.explanation and {'by_forecast_step': ...}.
  • An entry is the NIM's sdm_gradient_v1 record with row positions resolved: kumo-tabular gives
    {'outputs', 'instance'}; kumo-relational gives {'outputs', 'related'} with one block per
    table (the entity's own included), each row carrying its primary key when the table has one and
    its values as sent. The engine's question-row block is dropped. Positions are resolved per call,
    so split requests are correct.
  • result.explanations.to_pandas() flattens to one line per (row, table, hop, key, row, output,
    column, gradient); hop is computed from the relationships and keys sent.
  • Docs, skills, changelog and QA probes updated; unit tests with mocked NIM replies; live tests
    (-m live_nim) for both models pass.

API

Relational

result = client.predict(request, model=KumoRelational(),
                        outputs=['prediction', 'probabilities', 'explanation'])
result.explanations['user_5']
{'outputs': [...],
 'related': {'users':  {'keys': [...], 'rows': [...], 'columns': [...], 'gradients': [...]},  # [output][row][column]
             'orders': {'rows': [...], 'columns': [...], 'gradients': [...]}}}

As one table

result.explanations.to_pandas() gives one line per (row, output, column), across all entities:

id      table   hop  key     row  output           column       gradient
user_5  users   0    user_5  0    probability_yes  tenure_days   0.05
user_5  orders  1    None    0    probability_yes  amount        0.02
user_5  events  1    None    1    probability_yes  kind_login   -0.02

Tabular

result = client.predict(request, model=KumoTabular(),
                        outputs=['prediction', 'probabilities', 'explanation'])

result.explanations[901]            # keyed by the request's id column; by row_index if it has none
{
  'outputs':  [{'kind': 'probability', 'class': 0},
               {'kind': 'probability', 'class': 1}],
  'instance': {
      'columns':   ['age', 'spend'],
      'gradients': [[ 0.012, -0.004],   # probability_0 per unit age, spend
                    [-0.012,  0.004]],  # probability_1
  },
}

result.explanations.to_pandas()

id   table     hop  key  row  output         column  gradient
901  instance  0    901  0    probability_0  age      0.012
901  instance  0    901  0    probability_0  spend   -0.004
901  instance  0    901  0    probability_1  age     -0.012
901  instance  0    901  0    probability_1  spend    0.004

@JGagacheva
JGagacheva marked this pull request as draft October 12, 2026 23:57
@JGagacheva JGagacheva self-assigned this Oct 28, 2026
@JGagacheva
JGagacheva marked this pull request as ready for review October 29, 2026 22:43
Comment thread CHANGELOG.md Resolved
@JGagacheva
JGagacheva requested a review from Manushpm8 October 29, 2026 23:53
@JGagacheva
JGagacheva requested a review from adi-kumo October 30, 2026 20:30
JGagacheva added a commit that referenced this pull request Oct 30, 2026
P6 was stale (in-process supports explanation since #34), P3 named
only the entry points, P5 was silent on forecasts, D1 named a file
that does not exist.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@JGagacheva JGagacheva mentioned this pull request Oct 30, 2026
JGagacheva added a commit that referenced this pull request Oct 30, 2026
P6 was stale (in-process supports explanation since #34), P3 named
only the entry points, P5 was silent on forecasts, D1 named a file
that does not exist.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>

@Manushpm8 Manushpm8 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three earlier comments look fixed, thanks. One blocker: 9 commits lack a Signed-off-by, and AGENTS.md says unsigned PRs can't merge; git rebase --signoff main fixes it.

GraysonWu and others added 14 commits November 2, 2026 10:15
Cached Tabular predictions build a full envelope, including encoding
every context row, and then discard the context before sending the
request. Skip that encoding for session scoring while retaining schema
checks, pinned dtype conversion, and query serialization. One-shot
predictions and session creation still encode the context.

## Measured benefits

Mean one-row prediction latency fell from **467.4 ms to 52.4 ms**:
approximately **415 ms saved per call, 89% lower latency, or 8.9×
faster**. This improvement comes from removing repeated encoding of the
already-cached context.

## Benchmark configuration

- GPU: NVIDIA A10G; client and server on the same machine.
- Model: Kumo Tabular Large, BF16, one estimator.
- Task: fraud classification using a fitted session with a cached
context.
- Input: 10,000 context rows, 30 numerical features, one query row.
- Timed operation: `FittedModel.predict(...).to_pandas()`; context fit
and Request construction are outside the timer.

## Validation

- Same server and fitted session, two warmups followed by 20 alternating
measurements per version. This is a sequential latency comparison, not a
throughput or P99 claim.
- Payload contents matched excluding request IDs. Predictions were
bitwise identical for query batches of 1, 8 and 1,000 rows.
- Four regression cases covering classification/regression with and
without a pinned schema fail before the change and pass afterward.
Public API and all nine wire-payload snapshots are unchanged.
- Ruff, formatting, mypy, repository hygiene and attribution checks
passed for this patch.
- CI currently needs the compatibility and built-wheel test fixes in
#51. Verified this patch applied to #51 at
`f54b0938af0fcb12706b3a740304d26ae4837f0c` with pandas 3.1.0 and DuckDB
2.0.0: main suite 2,679 passed, separate engine suite 2,580 passed, and
1,963 client tests passed against the exact newly built wheel installed
by filename. Live-service tests were excluded; engine verification
reused the unchanged native extension. These are local combined-tree
results, not a green CI claim for this PR alone.

Signed-off-by: GraysonWu <graysonw@nvidia.com>
Co-authored-by: Manush Murali <91221099+Manushpm8@users.noreply.github.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
…#51)

## Summary

DuckDB 2.0.0 and pandas 3.1.0 broke CI on `main` and on every branch
based on it. #41 has since brought #47's holds (`duckdb<2`,
`pandas<3.1`) to `main`. This PR fixes what each release broke instead,
and takes the holds back out: no install caps either dependency, and
`uv.lock` differs from `main` only in those three specifiers.

- **DuckDB 2.0** warns that Arrow output from a DB-API result makes an
extra copy of the rows, and the engine's suite runs warnings as errors.
Two places now read through a relation,
`con.sql(query).to_arrow_table()`:
  - the engine's six DuckDB reads;
  - `read_table` over a caller's DuckDB connection.

DuckDB's warning names that call as the replacement, and 1.5 already has
it.
- **pandas 3.1** raises `OutOfBoundsDatetime` on `Timestamp.min -
DateOffset(0)`. A graph without a time column is anchored at
`Timestamp.min`, and a static query steps by `DateOffset(0)`, so every
relational prediction over such a graph failed.
- `shift_time` leaves a timestamp where it is when the offset moves
nothing. Every scalar anchor shift in `rfm.py`, and the base sampler's
window check, now go through it.
- The samplers' own shifts work on whole columns, which 3.1 handles, so
they are unchanged.
- **pandas 3.1** also builds a frame around an integer past the float
range, where earlier releases refused one. A request's description
relied on that refusal to leave out the table carrying it. `_wire_frame`
now refuses such a table itself, on every pandas release.

Every CI and release job now installs the wheels CI built by file name.
With `--find-links` plus a name and version, pip resolves PyPI's release
of that version as readily as the wheel built in the run, so a job could
test or describe the published 1.0.0 instead.

- **GitHub CI `build` dry-run, `installed-wheel` and `sbom`.** This PR
cherry-picked these from `586882c57`, then tightened them:
  - The pure-Python wheels are named in full.
  - The `build` job checks there is exactly one wheel per distribution.
- `installed-wheel` compares each distribution's `direct_url.json` with
`dist/` exactly.
- **The engine smoke test** in CI, in the release workflow and in the
internal pipeline. `PIP_FIND_LINKS` is replaced by a cibuildwheel
`before-test` that installs `_sibling_wheels/*.whl` by file name before
the engine.
- **The internal pipeline's `build` dry-run, `installed-wheel` and
`sbom`.** Its `installed-wheel` now makes the same `direct_url.json`
check.
- **The release SBOM.** The pure-Python wheels are installed by file
name, and the engine from `dist/` with `--no-index`.

Not in this PR: putting a development version on `main` between
releases. That changes how a release is cut, so it is a separate
proposal.

## Verification

- **Both dependency sets, run locally on Python 3.12 with #41 merged
in:** pandas 3.1.0 with DuckDB 2.0.0, and pandas 3.0.6 with DuckDB
1.5.6. On each:
- the engine suite passes 2588 tests, the client suite 1960, the
connectors suite 300 and the root suite 267;
- #41's in-process tests need PyTorch, so they skip locally; CI's engine
job runs them;
  - no run raises a DuckDB warning.
- **Each new test fails without its fix:**
- On pandas 3.1, a graph without a time column fails to build its task
table.
  - On DuckDB 2.0, `read_table` over DuckDB warns.
- A table holding a string and an integer past the float range was
described rather than left out; this one fails on both pandas releases.
- `shift_time` moves by `DateOffset(months=1, days=-31)`, an offset that
cancels on some dates.
- **The CI install commands, run locally against wheels built from this
branch:**
- Every first-party wheel comes from `./dist/` or `./_sibling_wheels/`.
  - The engine's exact pins are already met when it installs.
- The `direct_url.json` check refuses a PyPI install, and a `dist/`
directory somewhere else.
- **CI:** 20 of 20 checks pass on 215a1e6.
- The engine job runs the client suite with #41's in-process tests, on
PyTorch's CPU build: 2086 passed, none skipped.
  - The test jobs resolve pandas 3.1.0 and DuckDB 2.0.0.
- Every engine-wheels row installs its siblings from `_sibling_wheels/`
for all five Python versions, and none from PyPI.
- **Codex review:** the last round, on 215a1e6, found nothing.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Manush Murali <manushm@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from
4.3.1 to 4.3.4.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pypa/cibuildwheel/releases">pypa/cibuildwheel's
releases</a>.</em></p>
<blockquote>
<h2>v4.3.4</h2>
<ul>
<li>🔐 Warns when a uv build frontend is selected and the host uv is
older than 0.13.4, which still trusts the compromised eMudhra CA roots
(<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3021">#3021</a>)</li>
<li>🔐 Uses cibuildwheel's CA bundle for pip on Python 3.9, where pip
uses its bundled certifi, which still trusts the compromised eMudhra
roots (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3017">#3017</a>)</li>
<li>🔐 Uses cibuildwheel's CA bundle for pip on GraalPy, where truststore
is not supported (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3019">#3019</a>)</li>
<li>🔐 Uses cibuildwheel's CA bundle in Android, iOS, and Pyodide cross
venvs, where pip uses the host's OpenSSL CA file (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3020">#3020</a>)</li>
<li>🐛 Seeds build's isolated environment with the pinned pip on macOS
and Windows, fixing GraalPy 25.0 builds (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3018">#3018</a>)</li>
<li>💼 Stops Azure cache growth from <code>restoreKeys</code> (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3022">#3022</a>)</li>
</ul>
<h2>v4.3.3</h2>
<ul>
<li>🔐 Updates manylinux images and pip to fix vulnerability to <a
href="https://emudhra.we-have-a-problem.com/">eMudhra</a>.</li>
</ul>
<h2>v4.3.2</h2>
<ul>
<li>🔐 Updates our dependency pins to certifi==2026.10.19, following the
<a href="https://emudhra.we-have-a-problem.com/">eMudhra Intermediate CA
Compromise</a>. Linux builds may still be vulnerable through manylinux
images - a patch release will soon follow if so.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md">pypa/cibuildwheel's
changelog</a>.</em></p>
<blockquote>
<hr />
<h2>title: Changelog
ref: changelog</h2>
<h1>Changelog</h1>
<h3>v4.3.5</h3>
<p><em>27 October 2026</em></p>
<ul>
<li>🐛 Stops passing the target build environment to the host virtualenv
call, regression in v4.3.4 (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3023">#3023</a>)</li>
<li>🛠 Updates dependencies and constraints (pyodide-build 0.39.3,
CPython 3.15.2, etc) (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3024">#3024</a>,
<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3027">#3027</a>)</li>
<li>💼 Updates CI actions (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3025">#3025</a>)</li>
</ul>
<h3>v4.3.4</h3>
<p><em>22 October 2026</em></p>
<ul>
<li>🔐 Warns when a uv build frontend is selected and the host uv is
older than 0.13.4, which still trusts the compromised eMudhra CA roots
(<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3021">#3021</a>)</li>
<li>🔐 Uses cibuildwheel's CA bundle for pip on Python 3.9, where pip
uses its bundled certifi, which still trusts the compromised eMudhra
roots (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3017">#3017</a>)</li>
<li>🔐 Uses cibuildwheel's CA bundle for pip on GraalPy, where truststore
is not supported (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3019">#3019</a>)</li>
<li>🔐 Uses cibuildwheel's CA bundle in Android, iOS, and Pyodide cross
venvs, where pip uses the host's OpenSSL CA file (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3020">#3020</a>)</li>
<li>🐛 Seeds build's isolated environment with the pinned pip on macOS
and Windows, fixing GraalPy 25.0 builds (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3018">#3018</a>)</li>
<li>💼 Stops Azure cache growth from <code>restoreKeys</code> (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3022">#3022</a>)</li>
</ul>
<h3>v4.3.3</h3>
<p><em>21 October 2026</em></p>
<ul>
<li>🔐 Updates manylinux images and pip to fix vulnerability to <a
href="https://emudhra.we-have-a-problem.com/">eMudhra</a>.</li>
</ul>
<h3>v4.3.2</h3>
<p><em>20 October 2026</em></p>
<ul>
<li>🔐 Updates our dependency pins to certifi==2026.10.19, following the
<a href="https://emudhra.we-have-a-problem.com/">eMudhra Intermediate CA
Compromise</a>. Linux builds may still be vulnerable through manylinux
images - a patch release will soon follow if so.</li>
</ul>
<h3>v4.3.1</h3>
<p><em>10 October 2026</em></p>
<ul>
<li>🛠 Updates CPython 3.15 to the final 3.15.0 release (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3009">#3009</a>)</li>
</ul>
<h3>v4.3.0</h3>
<p><em>5 October 2026</em></p>
<ul>
<li>✨ Extends <a
href="https://cibuildwheel.pypa.io/en/stable/configuration/#inherit">option
inheritance</a> to every configuration layer, so global and
platform-specific settings can append or prepend to existing values,
including cibuildwheel's defaults. Environment variables can also extend
earlier settings using <code>CIBW_INHERIT</code>. (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/2928">#2928</a>)</li>
<li>✨ Adds PyPy 3.12 support and updates PyPy 3.11 to v8.0.0, available
with the <code>pypy</code> <a
href="https://cibuildwheel.pypa.io/en/stable/options/#enable"><code>enable</code></a>
flag (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/2998">#2998</a>)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pypa/cibuildwheel/commit/b9e566a2b284570af229f6c2a33673410d0389a3"><code>b9e566a</code></a>
Bump version: v4.3.4</li>
<li><a
href="https://github.com/pypa/cibuildwheel/commit/0d511ac4b39be40c67cc7c02a075e6296f1cf8d4"><code>0d511ac</code></a>
fix: use cibuildwheel's CA bundle in cross venvs (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3020">#3020</a>)</li>
<li><a
href="https://github.com/pypa/cibuildwheel/commit/d6c9b11a75d75d57f569bce12237551fb15a7f5a"><code>d6c9b11</code></a>
feat: warn when uv is too old to have the eMudhra CA fix (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3021">#3021</a>)</li>
<li><a
href="https://github.com/pypa/cibuildwheel/commit/44631207c5d0c25dad513048b972c5b112cc913e"><code>4463120</code></a>
fix: seed build's isolated env with the pinned pip (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3018">#3018</a>)</li>
<li><a
href="https://github.com/pypa/cibuildwheel/commit/6d4911a99e066f09dfa0a2228853ae4cfeefbc72"><code>6d4911a</code></a>
fix: use cibuildwheel's CA bundle for pip on GraalPy (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3019">#3019</a>)</li>
<li><a
href="https://github.com/pypa/cibuildwheel/commit/54d1a6f91a19d452909d450f78cb8d5909213ef2"><code>54d1a6f</code></a>
ci: stop Azure cache growth from restoreKeys (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3022">#3022</a>)</li>
<li><a
href="https://github.com/pypa/cibuildwheel/commit/5891574f2a763988a8d41520576fae65a641ac8c"><code>5891574</code></a>
fix: use cibuildwheel's CA bundle for pip on Python 3.9 (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3017">#3017</a>)</li>
<li><a
href="https://github.com/pypa/cibuildwheel/commit/8c2dcd2e96041a619f5ff03c9638d7941e2553a4"><code>8c2dcd2</code></a>
Bump version: v4.3.3</li>
<li><a
href="https://github.com/pypa/cibuildwheel/commit/56459e64a8cb22e00043efabcf53a4b0ecf38fc0"><code>56459e6</code></a>
[Bot] Update dependencies (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3012">#3012</a>)</li>
<li><a
href="https://github.com/pypa/cibuildwheel/commit/afd09e67071efbccba51f7690b2f5262bedcf535"><code>afd09e6</code></a>
chore(deps): bump the actions group with 4 updates (<a
href="https://redirect.github.com/pypa/cibuildwheel/issues/3013">#3013</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/pypa/cibuildwheel/compare/b3395e271a35f73dad77f21802abe9b986dfe299...b9e566a2b284570af229f6c2a33673410d0389a3">compare
view</a></li>
</ul>
</details>
<br />

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Adi Wadaskar <214436632+adi-kumo@users.noreply.github.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
## Summary

Snowflake attributes a session's queries by the application the session
names at login. Sessions the client opened named none, so they showed up
as the driver's default.

Each Snowflake session the client opens now names itself
`nvidia/kumo-sdm` when nothing else names it.

This was reviewed as #50, which merged into #51's branch rather than
`main`. #51 has reverted it, and it lands here on its own, with one fix
from review since.

- **The name is added after the driver resolves the connection.**
`_NamedConnection` subclasses the driver's `SnowflakeConnection` and
adds `application` in `connect`. The driver first calls `connect` once
it has merged three things:
  - the caller's arguments;
- the `connections.toml` profile, read from the caller's
`connections_file_path` if one is given;
- its own detection of `SF_PARTNER`, Streamlit, Jupyter and Snowflake
notebooks.
- **Any other name is kept.** That includes an `application` argument,
one in the profile, `SF_PARTNER`, or one of the driver's notebook tags.
- **Fixed since #50:** with no arguments, or only
`connections_file_path`, the driver replaces what it was given with the
default connection's parameters, which drops the name it had just
detected. The session was then named `nvidia/kumo-sdm` under Streamlit
or `SF_PARTNER`. `_NamedConnection` now asks the driver's detection
again before using its own name (e6e8992).
- **Everything else is the driver's own.** The client does not read the
driver's config or choose its default connection. The connections file,
the default connection and the error when nothing is configured all
behave as before.
- **Connections the caller owns are unchanged.** A borrowed Snowpark
session, or a connection the caller opens and passes to a source, is
used as it is.

## Tests

`test_snowflake_names_a_session_only_where_nothing_else_does` runs the
driver's real resolution (snowflake-connector-python 4.8.0) over a real
`connections.toml`, and stops at login. Its cases:

| Session opened with | Application sent |
|---|---|
| credentials | `nvidia/kumo-sdm` |
| credentials and `application='theirs'` | `theirs` |
| credentials, `SF_PARTNER` set | the partner |
| credentials, Streamlit loaded | `streamlit` |
| a profile that sets `application` | the profile's |
| a profile that doesn't | `nvidia/kumo-sdm` |
| only `connections_file_path` | the default profile, and
`nvidia/kumo-sdm` |
| only `connections_file_path`, `SF_PARTNER` set | the default profile,
and the partner |
| no arguments | the default profile, and `nvidia/kumo-sdm` |
| no arguments, Streamlit loaded | the default profile, and `streamlit`
|

What the cases catch:
- Against `main`'s backend, six fail: the four that should get
`nvidia/kumo-sdm`, and the two where the driver drops the name it
detected.
- Against #50 as reviewed, those last two fail.

Locally, the connectors suite passes (309 passed, 9 skipped), with ruff,
ruff format, mypy and the hygiene check clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Manush Murali <manushm@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Allow the explanation output for kumo-tabular (one scored row), return it on result.explanation, and drop the old 1,000-row context rule for relational explanations, which the SDM NIM does not have.

Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
The architecture table of what each model declares still marked
`explanation` as absent for kumo-tabular, and a session test still said
the endpoint only explains under the 'fast' run mode.

Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Every scored row is explained, on both models, and result.explanations
maps each row's key (id, (id, forecast_step), or row_index) to its
entry. The docs, skills and changelog drop the one-row limit and
result.explanation, describe the entry shape, related rows with keys
and values, preprocessed columns and to_pandas() with hop, and name the
server settings a NIM needs to explain (SDM_NUM_ESTIMATORS=1, and CUDA
graph replay off on kumo-tabular), which otherwise answer 422.
probe_tabular.py probes a multi-row explanation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
The NIM now explains every predict row, one explanation per record, so
the client no longer refuses an explanation over more than one row:
explanation_predict_rows leaves TABULAR_LIMITS and RELATIONAL_LIMITS,
and the tabular row check and the relational one-entity check go with
it. Tests that asserted those refusals are removed or now assert the
rows are sent, and the skills limits table no longer maps a row to the
removed limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
result.explanation kept the first record's explanation and dropped the
rest, and a forecast gathered its steps under {'by_forecast_step': ...}.
Both are replaced by result.explanations, a mapping from the key that
names a row in to_pandas() -- its id, (id, forecast_step) for a
forecast, or row_index when the request has no entity key -- to that
row's explanation, kept out of the frame and empty when none was asked
for.

Each entry is read beside the payload of the call that produced it, so
a split request's positions, which restart at 0 in every call, are
translated before the calls are merged. The instance block drops its
row_indices and its gradients become [output][column]. A related row
is given back as its primary key, when the table has one of its own,
and always as its values as sent; kumo-tabular entries have no related
key. Explanations.to_pandas() flattens every gradient to one line, with
each related row's hop from the entity worked out from the
relationships and key values sent. Explanations is exported, since
result.explanations hands it out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
The client refused an explanation whose format was not sdm_gradient_v1
with INVALID_RESPONSE. Both models answer in that one format, so the
check guarded nothing; the details are now read directly.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
kumo-relational sends the entity as two tables: the per-prediction
instance row (entity key, anchor time, engine bookkeeping) and the
entity's own feature row as a related table. The docs put the entity's
features under instance; they are under related['users'], and
to_pandas() shows the entity on an instance line and a users line, both
hop 0. The example tables now name each output by its result column
(probability_yes), and say what key and hop hold on an instance line and
for an unreached row.

The relational probe for an explanation over several entities expected
a refusal before the wire; it now checks one entry per scored entity,
keyed by id, with related tables.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Tabular: every scored row is explained under its id, or its row_index
when the request has no key; the instance gradients are the row's own,
[output][column]; an entry has no related key; outputs and columns are
kept as returned.

Relational, on payloads shaped as the engine gathers them and answers
shaped as the NIM user guide gives them: related positions are read as
that call's rows, a keyed table's keys come back as their declared
dtype (a key past 2**53 included) while rows keep the values as sent,
a keyless (__node_id) table has no keys, and gradients line up with
rows. A request split into calls that each number from 0 keeps every
entity's own rows, and row_index keys move with their rows. A forecast
is keyed (id, forecast_step).

Explanations.to_pandas(): its columns, one line per (row, table,
related row, output, column), hop 0/1/2 through a two-hop graph and
empty for an unlinked row, and outputs named as the frame's columns.

The NIM explanation builder is a shared conftest fixture, and
tabular_request can build a request with no key column.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
The parity case expected a kumo-relational NIM to refuse an explanation
with UNSUPPORTED_OUTPUT, which a NIM that explains every row no longer
does. It is removed; the in-process suite now checks that a model run
in this process refuses 'explanation' for both models.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
JGagacheva and others added 19 commits November 2, 2026 10:15
The live fixture graph had only keyed tables, so the keyless case of the
explanation contract (rows without keys, no __node_id, null key in the
flat table) was never checked against a NIM. Add events, linked to
users.user_id with no primary key, and check its blocks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Rename the result class Explanations to GradientExplanations, and move
explanations_of and the relational adapter's related-row helpers into one
GradientDecoder in wire/explanations.py, built once per call with the
call's payload (kumo-relational) or none (kumo-tabular). No behaviour
change. The public API snapshot is regenerated for the rename.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
The kumo-relational NIM sends a related table's primary key only when a
relationship joins on it, so orders blocks carry rows without keys, and it
now serves explanation for every relational target.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Both were public in 1.0.0. result.explanation returns the first record's
raw explanation, or None, and PredictionResult(explanation=...) is
accepted again; each raises DeprecationWarning naming result.explanations
and removal in 2.0.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
JGagacheva added a commit that referenced this pull request Nov 2, 2026
P6 was stale (in-process supports explanation since #34), P3 named
only the entry points, P5 was silent on forecasts, D1 named a file
that does not exist.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Comment thread tests/test_skills.py Resolved
@JGagacheva
JGagacheva merged commit 1c5e2e7 into main Nov 3, 2026
20 checks passed
@JGagacheva
JGagacheva deleted the jg-enable-explain branch November 3, 2026 18:49
JGagacheva added a commit that referenced this pull request Nov 3, 2026
P6 was stale (in-process supports explanation since #34), P3 named
only the entry points, P5 was silent on forecasts, D1 named a file
that does not exist.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants